Category-effects and stimulus characteristics in visual perception

......................................................................................... 6 INDRODUCTION ................................................................................. 7 Category effects in object identification – Deficits after brain damage .............. 8 Category effects in object identification – The 'normal' living-things advantage ... 11 Visual characteristics of stimuli in object-related category specific effects .......... 12 "Structural similarity" and the "global" or outline shape of objects ................... 15 Outlines and "global" shape in category specific effects ................................. 17 Eye-movements and category-specificity in object identification ..................... 19 Category specificity in colour perception ................................................. 20 METHODOLOGICAL CONSIDERATIONS ................................................ 23 Controlling stimuli for "nuisance" variables ............................................. 23 Effects specific to a stimulus set? ....................................................... 25 Computational modelling of category-specific effects ................................. 26 GENERAL RESEARCH QUESTIONS ....................................................... 29 COMMON METHODS AND DESIGNS ...................................................... 30 SUMMARY OF RESEARCH REPORTS ..................................................... 31 Report I ....................................................................................... 31 Report II ....................................................................................... 32 Report III ...................................................................................... 33 Report IV ...................................................................................... 35 OVERALL CONCLUSIONS ..................................................................... 37 REFERENCES ....................................................................................... 40 RESEARCH REPORTS ............................................................................ 51 Category effects in visual perception 3 This thesis is dedicated to all good reductionists. If you read trendy intellectual magazines, you may have noticed that reductionism is one of those things, like sin, that is only mentioned by people who are against it. To call oneself a reductionist will sound, in some circles, a bit like admitting to eating babies. R. Dawkins, The Blind Watchmaker Reductionism, like cholesterol, comes in good and bad forms. [...] Good reductionism (also called hierarchical reductionism) consists not of replacing one field of knowledge with another, but of connecting or unifying them. The building blocks used by one field are put under the microscope of another. The black boxes get opened; the promissory notes get cashed. Steven Pinker, The Blank Slate Category effects in visual perception 4 ACKNOWLEDGMENTS The research presented in this thesis was supported by grant 148218/330 from The Research Council of Norway. It was carried out at the Department of Psychology, University of Tromsø, Norway. I wish to express my most deep-felt gratitude to my main supervisor, Professor Bruno Laeng. Not only for his invaluable help and advise in the work underlying this thesis, but also for letting me participate in a number of exciting research projects, and for introducing me to several researchers in the field. He has shown me how to present, how to review, how to redo (experiments), and how to revise. Most of all, I am grateful to him for always being a source of inspiration and support – in short, for being a model scientist worthy of emulation by any aspiring researcher. I am also very grateful to co-supervisors Professors Tim Brennen and Michele Miozzo. Professor Brennen has, since my time as undergraduate student, given unstinting and precious support, encouragment and guidance. Professor Miozzo was an exceptionally generous host and an inspiring tutor and supervisor during my stay at Columbia University in 2003. I also wish to thank my colleague Ph.D.-students and other employees at the Department of Psychology in Tromsø for providing me with a suitable infrastructure and a good environment in which develop as a student, researcher, teacher, and as a person. Last, but most warmly of all, I wish to thank my life's companion, Camilla, for innumerable things, great and small, many of which have contributed to my being able to carry through with my first major research project, and for all of which my gratitude defies expression. Tromsø, July 25, 2006 Torstein Låg Category effects in visual perception 5 RESEARCH REPORTS The present dissertation is based on the following empirical reports: Report I Låg, T. (2005). Category-specific effects in object identification: What is 'normal'? Cortex, 41, 833-841. Report II Låg, T., Hveem, K, Ruud, K.P.E., & Laeng, B. (2006). The visual basis of category effects in object identification: Evidence from the visual hemifield paradigm. Brain and Cognition, 60, 1-10. Report III Låg, T, & Laeng, B. (submitted). Eye-position during identification of living and nonliving objects seen in canonical and non-canonical views. Report IV Laeng, B., Låg, T., & Brennen, T. (2005). Reduced Stroop interference for opponent colors may be due to input factors: Evidence from individual differences and a neural network simulation. Journal of Experimental Psychology: Human Perception and Performance, 31, 438-452. Category effects in visual perception 6 ABSTRACT This thesis describes a number of experiments that aimed to investigate the role of relatively low-level visual input factors in category-specific effects in object identification and colour perception. In the object recognition experiments, using picture-name or name-picture verification tasks, as well asThis thesis describes a number of experiments that aimed to investigate the role of relatively low-level visual input factors in category-specific effects in object identification and colour perception. In the object recognition experiments, using picture-name or name-picture verification tasks, as well as object-naming tasks, clues to the causal factors contributing to such effects were obtained. It was found that category-specific effects in normal object identification, both living things advantages and living things disadvantages can occur even when nuisance variables like familiarity and complexity are well controlled. Task demands on perceptual differentiation and stimulus presentation conditions can influence and even reverse category-specific effects (Report I). When identification has to rely mostly on global shape visual information, the living things advantage in identification is enhanced compared to when visual detail is available in stimulus pictures. Furthermore, a lack of visual detail induces a left hemisphere disadvantage for identification, but only for nonliving things (Report II). In an experiment utilising eye movement methods, it was found that when rotating objects in depth, which presumably causes changes in outline shape, changes in participants' eye movement strategies could be observed. Specifically, participants tended to focus more on the objects' centres of gravity when rotations went from canonical to noncanonical views. This effect was, however, only reliably observed for nonliving things. (Report III). In a study examining differential interference effects in Stroop performance, it was found that the amount of interference is smaller for non-opponent compared to opponent colours. An artificial neural network that coarsely implements a trichromatic input coding scheme can simulate this reduced opponent colour interference. Additionally, it was found that individual differences in colour discrimination ability are associated with individual differences in Stroop performance. (Report IV). Category effects in visual perception 7 INTRODUCTION Within the cognitive sciences, we can define category specific effects as performance differences in visual perception tasks that are related to, or dependent on, the semantic category to which a group of stimuli belongs. For instance, for object stimuli (typically pictures/drawings of common objects), a category specific effect would manifest itself in more efficient recognition or identification performance for objects from one domain or category (e.g. socalled living things, such as horses, oranges, birds, and carrots) compared to another domain (e.g. nonliving things, such as hammers, aeroplanes, chairs, and microscopes). Category specific effects in object identification will be examined specifically in Papers I, II, and III of the present thesis. Regarding another stimulus class, colours, a category specific effect can manifest itself in more efficient colour naming for colours belonging to a specific colour category (e.g. focal colours versus non-focal colours) or in different magnitudes of interference in a colour-word/ink-colour Stroop (1935) task. This latter kind of category specific colour perception effect is dealt with in Paper IV of this thesis. Category specific effects are curious phenomena, and, as such, well worth a study in their own right. However, I would also suggest that they provide a unique window onto the perceptual processes that underlie the tasks in which such performance differences are observed. The reasoning behind this contention is relatively straightforward, and might be sketched as follows: Given a task, T, which is basically identical for all the stimuli on which it is performed, and given a process, P, assumed to underlie performance on T, assume that two groups of stimuli, a and b, are presented to research participants for execution of task T. If performance on task T differs systematically depending on whether participants are executing it on stimulus group a or b, then, given the assumption that P underlies T, we now know that there are (or is) some crucial properties (or property) that differ(s) between the stimuli in a and b and that somehow affect(s) the workings of P. Thus, pinpointing these properties will give Category effects in visual perception 8 important clues to which factors affect the workings of P. And, revealing the workings of the processes underlying perceptual tasks is, arguably, the raison d'être of visual cognitive science. Category effects in object identification – Deficits after brain damage Among sub-topics in the field of category specific effects in visual perception, there is no doubt that the study of deficits in object identification after brain damage is the dominant one. It is also the sub-field that started the study of category specific effects. Early case descriptions provided by Elizabeth Warrington, Alfonso Caramazza and others (Hart, Berndt, & Caramazza, 1985; Hillis & Caramazza, 1991; Warrington & McCarthy, 1983, 1987; Warrington & Shallice, 1984) represent a starting point for the systematic study of this kind of category-specific deficit. In these early studies, patients displaying deficits in picture naming or semantic feature assignment characterised by the selective impairment in the processing of either natural objects/living things (such as animals and plants) or artefacts/nonliving things (typically man-made objects, such as tools, furniture, vehicles etc) were described. For instance, two patients, SBY and JBR, examined by Warrington and Shallice (1984) showed a remarkable preservation of the ability to identify nonliving things relative to living things and foods. In one experiment, these patients were given a set of coloured pictures of objects to name or describe. In the nonliving category (containing items like car, mop, chair, axe) they identified 90% and 75% of the objects respectively, whereas in the living things category (containing items like deer, crocus, lizard, sheep) they managed only 6% and 0%. A rather impressive number of similar cases have now been described (see Capitani, Laiacona, Mahon & Caramazza, 2003 for a recent review), along with (considerably fewer) cases showing the opposite pattern of deficit; that is, relative preservation of living things compared to nonliving things (see Laiacona & Capitani, 2001, for a discussion of such cases). Although there are unsolved Category effects in visual perception 9 methodological problems concerning performance levels of cases and control subjects, as well as adequate task and stimulus controls (Laws, 2005), most researchers working on category specific deficits have taken the collective data to imply a double dissociation of the processing of living and nonliving things. A double dissociation is roughly defined as the situation in which a patient A performs well on task X but is impaired on task Y, while another patient B is impaired on task X but performs well on task Y (see Shallice, 1988 for a more elaborate definition, and Laws, 2005 for a critique of the notion that the case-literature on category specific deficits contains pairs of cases that qualify as double dissociations). Double dissociations are generally thought to imply the existence of two separate processing or representational systems, mechanisms or modules. In the case of the living/nonliving dissociation, this would imply separate conceptual stores or modules; one for living object concepts, and another for nonliving object concepts. There are existence proofs from computational cognitive neuroscience, however, that unitary systems sometimes behave in ways that constitutes a double dissociation (e.g. Devlin, Gonnerman, Andersen & Seidenberg, 1998; Plaut, 1995) without it actually arising from two systems. Thus the two-system account of the living/nonliving dissociation is not as straightforward as it might initially seem. Indeed, apart a more sophisticated reincarnation of this type of explanation (see Caramazza, 1998; Caramazza & Mahon, 2003; Caramazza & Shelton, 1998; Shelton & Caramazza, 2001), explanations of the living/nonliving dissociation have taken a variety of different approaches. The plethora of theories put forward to explain this double dissociation is much too extensive to be reviewed here. It is, however, worth noting that most of them are couched in terms of the organisation and structure of semantic or conceptual memory (in Laiacona et al.'s, 2003 review, the authors conclude that the results of studies claiming a non-semantic locus of the deficit are not reliable). Only some authors have made attempts to link explanations of Category effects in visual perception 10 category specificity to the visual characteristics of the stimulus objects. The absence of this kind of explanation is perhaps surprising given the particular emphasis it was given by Elaneaor Rosch in parts of her work (Rosch, Mervis, Gray, Johnson & Boyes-Braem, 1976). However, if one considers the bulk of the work that has been done on concepts in cognitive psychology, it does seem to lean toward abstract, non-sensory features and processes as regards the terms in which it couches its hypotheses and explanations (see e.g. Murphy, 2002). Nevertheless, linking category-specific deficits to visual features has been attempted. Specifically, the role of structural or visual similarity has been the focus of some attention (e.g. Forde & Humphreys, 2001; Gale, Done & Frank, 2001; Humphreys, Lamote & Lloyd-Jones, 1995). The first notable attempt to link structural similarity to category effects was presented in Humphreys, Riddoch and Quinlan's (1988) "cascade theory" of picture naming. (The "cascade theory" is so named for its emphasis on the gradual, cascading spread of activation from early to later processing stages.) These authors hypothesised that one possible reason for the disproportionate number of cases displaying a living thing deficit (as opposed to a nonliving thing deficit) might be the higher "within-category structural similarity" of living things. (Structural similarity was operationalised as the percentage of contour overlap and the number of subjectively rated common parts of linedrawn figures of objects). Specifically, they suggested that since living categories, in particular animals, typically have lots of structurally similar object types that is, they are "visually crowded" (e.g. a horse looks like a donkey looks like a zebra looks like an elk etc.) they require a more detailed visual analysis of the objects' overall shape to keep from mixing them up (Forde & Humphreys, 1999). If the system that deals with or stores information about visual structure is damaged, living things will be more vulnerable to processing deficits than nonliving things. These authors have found support for their proposal in findings of a tendency for neurologically intact participants to be Category effects in visual perception 11 slower and less accurate at identifying living things (e.g. Capitani, Laiacona, Barbarotto & Trivelli, 1994; Gaffan & Heywood, 1993; Lloyd-Jones & Humphreys, 1997), and in certain measurements that seem to confirm the assumption of a higher structural/visual similarity within living things categories (Humphreys et al., 1988). Category effects in object identification – The 'normal' living-things advantage The idea (and the findings supporting it) that there is a "normal asymmetry" (Capitani et al., 1994) in object identification, with less efficient processing of living things compared to nonliving things, was challenged in 1999 by a report of the opposite effect (Laws & Neve, 1999); that is, more efficient naming of living compared to nonliving things. Similar findings have since then appeared in other reports (e.g. Gerlach, 2001; Laws, 2000), and thus the living things advantage is unlikely to be a fluke. In fact, it has been suggested that it is the findings of the living things disadvantage in the older reports that might benefit from reinterpretation. As Laws and Neve (1999) correctly point out, the early studies of category specific effects in normal participants did not control across stimulus category for potentially confounding variables (see Funell & Sheridan, 1992; Stewart, Parkin & Hunkin, 1992) such as concept familiarity and visual complexity (cf. Snodgrass & Vanderwart, 1980). Still, perhaps the most intriguing aspect of the newer reports of a living things advantage is the fact that their authors take an approach to explaining the effect that is conspicuously similar to the structural similarity hypothesis of Humphreys and colleagues (Humphreys et al., 1988), despite the fact that they are essentially attempting to explain the inverse effect. Laws and Neve (1999), for instance, suggested that exemplars within living things basic level categories tend to be more structurally similar to each other (e.g. any given horse resembles any other horse to a relatively high degree) than exemplars within non-living Category effects in visual perception 12 categories (consider for instance the variety of shapes that different chairs can have). This kind of higher structural similarity tends to make the visual representations of living things more stable and informative with regard to their basic level identity, and so an advantage in identification is the result. Thus, both the explanation of the category specific disadvantage for living things, and the explanation of the category specific advantage for living things appeal to the structural/visual similarity of the to-be-identified objects. These two possibilities really seem mutually exclusive, but as Gerlach (2001; Gerlach, Law & Paulson, 2004) has pointed out, they may not be. The key to seeing this is to note that the structural similarity of Humphreys and colleagues' hypothesis (Humphreys et al., 1988; 1995) concerns comparisons between basiclevel objects (e.g. horse, zebra, donkey) within higher-level categories (e.g. animals), whereas the structural similarity of Laws and Neve (1999) concerns comparisons between exemplars or sub-types of objects (e.g. Shire horse, Arabian horse, Shetland pony etc.) within a basic level (cf. Rosch, Mervis, Gray, Johnson, & Boyes-Braem, 1976) object class (e.g. horses). One suggestion is that the two different-level similarities affect object identification more or less depending on task and stimulus parameters. Two questions that arise, then, are: (i) Is poor stimulus-control the real explanation for the living things disadvantage in normal object identification reported in Humphreys and other's work (Capitani, Laiacona, Barbarotto & Trivelli, 1994; Gaffan & Heywood, 1993; Humphreys, et al., 1988; Lloyd-Jones & Humphreys, 1997)? (ii) If not, what are the conditions under which the advantage or disadvantage for living things occur? Visual characteristics of stimuli in object-related category specific effects As I mentioned above, attempts to explain the category specific deficits that some patients display after brain damage have overwhelmingly been couched in terms of the organisation of semantic object knowledge. Indeed, Category effects in visual perception 13 support for this type of explanation has also been pursued in experimental studies using neurologically intact participants as informants. The work of Helen Moss and Lorraine Tyler (Moss, Tyler & Devlin, 2002; Tyler & Moss, 1997; 2001; Tyler, Moss, Durrant-Peatfield & Levy, 2000) as well as other researchers (e.g. Cree & McRae, 2003; Garrard, Lambon-Ralph, Hodges & Patterson, 2001; McRae, de Sa, & Seidenberg, 1997) is representative. In these explanations, the focus is on the patterns of correlation and distinctiveness among features of different types of object. Despite this "semantic dominance", there are several considerations that make it seem worthwhile to pursue an apparently rather different type of explanation, focusing on the visual characteristics of objects, rather than the organisation of our concepts of them. However, this is likely to be less controversial for the category specific effects observed in normal object identification than for deficits after brain damage (cf. Capitani et al., 2003). As the work in the present project has focused on category-effects in neurologically intact participants, the following discussion will focus on what is most relevant to these. The above-mentioned considerations can be briefly sketched as follows: First, the speculations of Laws and Neve (1999) and of Gerlach (2001) concerning the visual similarity within exemplars of basic level object classes seem plausible. Although one might also rather easily come up with counterexamples, it seems that the positive examples are more numerous. An illustration of their main point is provided in Figure 1, which shows exemplars of the basic level object classes 'cat' and 'chair'. As is evident from this example, the general shapes of different cats are far more similar to each other than the general shapes of chairs. As we will see later, it may be important that this similarity apparently is already present in the outline shape of the objects, and that the details of internal patterns may be irrelevant. Category effects in visual perception 14 Figure 1. Exemplars of the basic level object classes 'cat' and 'chair' exemplifying Laws and Neve's (1999) speculation that living things may have higher visual similarity within basic level categories than nonliving things. Second, there has recently been put forth proposals that seem to open up a bit the bulkheads that have previously separated research on object recognition and perceptual representations on the one hand, and categorisation and representations of object concepts on the other (see Palmeri & Gauthier, 2004 for a review). Notably, in the work of Barsalou and colleagues (Barsalou, 1999; 2003; Goldstone & Barsalou, 1998; Pecher, Zeelenberg & Barsalou, 2004; Solomon & Barsalou, 2001; 2004), the dependency of abstract semantic knowledge on specific episodes of perception is emphasised and documented. Within these accounts, conceptual or featural similarity, which is so crucial in accounts of concept representations, is grounded in perceptual representations, which can implicitly represent properties in an analogue fashion and provide impressions of overall similarity. Following this line of thought, one might surmise that even though many category effects seem to be best described as semantic effects, they may ultimately have arisen on the basis of perceptual processes and the visual characteristics of the objects themselves. Category effects in visual perception 15 "Structural similarity" and the "global" or outline shape of object representations A third consideration that makes it seem fruitful to pursue evidence of the influence of visual characteristics of objects in category effects is work in the field that perhaps could be expected to be most relevant to the question of category effects, viz. the field of object recognition itself. In order to see the relevance of this work for category specificity, it is important to consider one question: What is meant by "structural similarity" (e.g. Humphreys et al., 1988)? As I briefly mentioned above, Humphreys and colleagues (Humphreys et al., 1988) operationalised "structural similarity" in two different measures: In one measure, they asked independent participants to rate the 'number of parts' that exemplars of a given category shared. This measure is liable to have relatively modest reliability, given the subjective nature of the informant's decisions concerning what should count as an object part. Nevertheless, this measure was found to correlate rather strongly with the more objective second measure of structural similarity: 'contour overlap'. For this measure, Humphreys and co-workers normalised the line-drawings from the Snodgrass and Vanderwart (1980) set for size and orientation. They then overlaid each object picture in a category on each other object picture from the same category, and measured, by means of a grid, the contour overlap of each object pair. In this way, an average contour overlap for each category was obtained. Although it is not immediately obvious what is actually measured by the rated number of common parts, it seems fairly clear that the contour overlap measure reflects the outline of the objects' figures as they appear in the 2D drawings. Thus, one reasonable interpretation of the notion of structural similarity among objects seems to be similarity of the objects' outline shapes. The idea that outline shape may play a role in category effects is plausible in light of the relevance given by cognitive scientists (e.g. Marr, Biederman) to outlines in visual shape perception and object recognition. Early evidence of the importance of outlines in object recognition was provided Rock, Halper and Category effects in visual perception 16 Clayton (1972). These researchers investigated participants' immediate memory for figures made complex by the inclusion of internal details that were essentially irrelevant to the overall shape of the figure. Memory for such figures was poor compared to memory for simpler shapes. The authors conclude, "Whichever components of a complex figure are immaterial with regard to its global shape" (Rock et al., 1972, p. 672), will not be adequately apprehended during single exposures, and therefore fail to establish memory traces. The theoretical and empirical work of Donald Hoffman, Manish Singh and others assigns a number of crucial roles to the outer contours or silhouettes in visual shape perception (e.g. Feldman & Singh, 2005; Hoffman & Richards, 1984; Hoffman, 1998). They have, for instance, provided evidence that local minima of curvature in silhouettes (i.e. concave curves in shapes) provide the basis for parsing shapes into parts (Hoffman & Richards, 1984; Hoffman & Singh, 1997), that local minima-derived part boundaries play a role in the perception of transparency (Singh & Hoffman, 1998), and that properties of contours underlies our ability to see partially occluded objects as whole (Singh, Hoffman & Albert, 1999). That contours or figure boundaries are crucial in accounts of higher-level object perception and identification as well becomes apparent when one browses through the literature on theories of object recognition. For instance, arguments in support of the recognition-by-components view of Biederman (1987) are based in part on evidence that deletion of object's contours can affect their recognition when the local minima of curvature (i.e. the parts of contours that underlie parts parsing of shapes cf. Hoffman and Richards, 1984) are deleted. Thus, in a sense, the idea that object recognition is parts-based is founded on the realisation that certain segments of shape contours provide more important information about object parts than others (cf. Biedrman & Bickle, 1985, discussed in Biederman, 1987). Also, in the more recent work by Hayward and colleagues (Hayward, 1998; Hayward, Tarr & Corderoy, 1999; Keane, Hayward Category effects in visual perception 17 & Burke, 2003) the role of outlines of objects in their recognition is highlighted by results showing that recognition of objects composed of simple volumetric components, when viewed across depth rotations, is no worse for silhouettes than it is for shaded images, and that changes in outline shape predict changes in recognition performance. Outlines and "global" shape in category specific effects It takes no great leap of imagination to derive the possible role of outlines or global shapes in category-specific effects on the basis of the evidence concerning contours and silhouettes in shape and object perception sketched above. Indeed, in an experiment by Vannucci, Viggiano and Argenti (2001), participants identified images of living and nonliving objects. These images were spatially low-pass filtered at nine different levels of resolution. (Low-pass filtering removes high-spatial frequencies, thus leaving only the lower spatial frequencies. This causes a blurring of internal details, making them blend and become indistinguishable, whereas the outer borders will be less affected. The information provided by low-pass filtered images is thus primarily information about the global shape properties of the depicted object. Examples of low-pass filtered images are provided in Figure 2.) This made it possible to determine the identification threshold for each of three categories. Results showed that animals were on average identified at a lower level of resolution than tools and vegetables, indicating that information provided mainly by the global shape of animals reveals more of their identity, whereas the global shape of tools is less helpful to their identification, in line with the speculations of Laws and Neve (1999) and Gerlach (2001). Unfortunately, in this experiment, stimuli were not matched across category for potentially confounding variables like familiarity and complexity (see section on methodological concerns, below), making interpretation of the results less straight forward. Category effects in visual perception 18 Figure 2. Examples of low-pass filtered images in which internal details are obscured but the outline clearly discernable. Lloyd-Jones and Luckhurst (2002) provided results that point in a similar direction. In their experiment participants were presented with line drawings of real and nonsense objects, and were required to decide whether a given drawing depicted a real object or not. In one condition, participants were presented only with the object's (or nonsense object's) silhouette, whereas in the other condition the objects were presented as normal line-drawings complete with internal details. Their results showed a general advantage for living things, but, remarkably, this advantage was significantly larger for the silhouettes. This seems to imply that when mainly outline or global shape information is available, living things are identified more easily than nonliving things. LloydJones and Luckhurst (2002), suggested that there is less useful information in nonliving things' outline contours. These findings, particularly if bolstered by similar results from experiments using different paradigms and stimuli, or even different overall methodological approaches, would strongly suggest outline or "global" shape as a causal factor in the category specific identification advantage typically observed in neurologically intact people. Category effects in visual perception 19 Eye movements and category specificity in object identification The study of eye movements to glean insight into various cognitive processes is an approach that has become rather popular in recent years. It is, however, not uncomplicated, given the relative multitude of factors that may influence eye movements (see e.g. Henderson & Hollingworth, 2003; Senders, Fisher & Monty, 1978). Even so, a few basic assumptions about eye movements seem to be rather uncontroversial. In the words of Henderson and Ferreira (2004), "eye movements provide an unobtrusive, sensitive, real-time behavioural index of ongoing visual and cognitive processing" (p. 18). But how do visual and cognitive processes manifest themselves in eye movements? Answers to this question are likely to vary according to the experimental paradigms used and the cognitive processes studied. However, it is generally assumed, and reasonably well documented, that our eyes tend to seek out informative regions of visual displays (Henderson & Hollingworth, 1998; 1999; Loftus & Mackworth, 1978; Mackworth & Morandi, 1967; Yarbus, 1967). What qualifies as an informative region may not be obvious, and in any case seems to vary somewhat depending on task and stimulus conditions (Parkhurst, Law & Neibur, 2002; Underwood, 2005), but both visual and semantic saliency appear to be likely candidates. One frequently reported phenomenon in eye movement research is the socalled "global" or "centre-of-gravity" effect (Coren & Hoenig, 1972; Findlay, 1982). The global effect is, roughly, that the end point of saccades tends to land near the centre of gravity of the visual displays used in a given task. The effect seems rather robust, and occurs for a variety of tasks and visual displays (e.g. Kowler & Blaser, 1995; McGowan, Kowler, Sharma & Chubb, 1998; Vishnawath & Kowler, 2003; 2004). However, and crucially, there is also evidence that the effect can be modulated by the demands of the task (Coëffé and O'Regan 1987; He & Kowler, 1989; 1991). He and Kowler conclude that the global effect may be an effective default strategy for information extraction Category effects in visual perception 20 that can be modulated or overridden by a preceding voluntary selection process when task demands require it. When observers are instructed to look at an object as a whole, they tend to fixate its centre of gravity (He & Kowler, 1991). They can, however, fixate other parts of the display if the task specifically requires it. If it is the case that the information provided by the visual representations of objects vary according to category (as has been hypothesised – see discussion above), then one might expect the global effect to be modulated accordingly. Specifically, given that the global effect is likely to be an expression of a strategy aimed at obtaining information from the whole of a visual display or object, and given the importance of the contours or silhouette of an object for its recognition (se pages 15-17), on might expect the increased reliance on the global effect strategy as the shapes presented by objects become more challenging. This again would imply increased reliance on the global effect strategy with nonliving objects. Category specificity in colour perception Colour perception as such is not the main focus of the investigations reported in this thesis. However, category effects, and the possibility that they arise from basic visual characteristics of the physical stimuli to be perceived, apply to the domain of colour as well. One peculiar colour-perception phenomenon, particular to some colours or colour combinations, might indeed be seen as category specific. This is the phenomenon of colour opponency, first described by the physiologist Ewald Hering toward the end of the 19 century (Hering, 1964). Hering noticed that we frequently perceive hues that seem to be mixtures of focal colours. For instance, we often perceive greens that are bluish, reds that are yellowish, or blues that are reddish. Some colour combinations are, however, never perceived. No one ever experiences a bluish yellow or a reddish green. On the basis of (among other things) this observation, Hering challenged the then dominant colour-perception theory of Helmholtz, by proposing the first Category effects in visual perception 21 colour-opponent receptor theory. Although the evolutionary or computational reasons for the development of colour opponency is not well understood (but see Purves & Lotto, 2002), the fact of its implementation in human physiology now seems accepted (e.g. De Valois & De Valois, 1993). As discussed above, the visual similarity of object's outlines or shapes has been implicated in the category specific effects observed in object identification. Interestingly, there are indications that similarity is also a factor in accounting for certain colour-perception phenomena. In an intriguing study by Dale Klopfer (1996), using the Stroop (1935) colour-naming interference paradigm (where participants are asked to name the ink colour in which a colour word is printed), it was found, not only that ink colours in incongruent colour-word pairs (e.g. 'blue' printed in green ink) were named slower than ink colours in congruent colour-word pairs (e.g. 'blue' printed in blue ink) – which is the normal Stroopeffect – but that incongruent interference was stronger with colour-word pairs containing similar colours (e.g. 'blue' in purple) compared to colour-word pairs containing dissimilar colours (e.g. 'blue' in yellow). Previous Stroop-studies have also found differential interference effects for different kinds of colour-word pairs. For instance, Klein (1964) varied the "semantic power" of the association between the word and the ink colour it was printed in, seeing more facilitation for 'blue' in blue than for 'purple' in blue, and slightly less again for 'sky' in blue. Here too there is a parallel to category effects in object identification: Explanations for this kind of colour-word pair interference effects are typically couched in terms of higher-order conceptual mechanisms. Seymour (1977) places the locus of Stroop interference at the conceptual encoding stage, and Stirling (1979) concludes that a combination of conceptual encoding and response competition are the most likely mechanisms for interference. It is perhaps not surprising therefore, that Klopfer (1996) also focuses more on higher-order processes than on lower-level ones, even though his colour-word pairs differ along a dimension (colour-similarity) that is Category effects in visual perception 22 intuitively rather low-level. Klopfer interprets his findings as evidence that the associative strength between words and colours affect the amount of interference (c.f. Klein, 1964), and concludes that they are thus consistent with the idea of Seymour (1977) that some of the interference arise at a conceptual level. Again, however, there may be good reasons to examine the possibility of Stroop-interference effects arising at a different level. First, and generally, effects arising at a semantic level can also have a visual sensory basis, if one allows that our concepts are derived from perceptual experience (c.f. Barsalou, 1999; Goldstone & Barsalou, 1998). Second, if colour-opponent cells have arisen because of a need to separate the correlated inputs of the retinal cones (c.f. Palmer, 1999), then the perceived dissimilarity of opponent colours (i.e. red-green and blue-yellow) and the perceived similarity of other colour pairs, may have a sensory basis. By extension, even though it may be true that the locus of Stroop-interference is conceptual, the organisation of colour-conceptual space that underlies this interference may well be determined by visual input factors. One could provide evidence for this line of reasoning if one could show, for instance, that low-level visual input factors affect Stroop-interference, that colour-word pairs with opponent and non-opponent colours yield different levels of interference, and that these effects could be implemented computationally by the manipulation of input factors in a neural network. Category effects in visual perception 23 METHODOLOGICAL CONSIDERATIONS The methodological concerns when studying the phenomena of visual perception are, needless to say, numerous. However, besides those issues that apply to almost all research in this field (e.g. using response times as a dependent variable (Miller, 1991; Posner 1986), whether or not to do ANOVA's by item as well as by subjects (Raajimakers, Schrijnemakers, Gremmen, 1999), and when not to use repeated measures ANOVA (Bagiella, Sloan & Heitjan, 2000; Quené & van den Berg, 2004)), there are some issues that are particularly pertinent in the study of category specific effects. In this section, I will discuss two problems that need to be dealt with in experiments on category effects in object recognition. I will, however, also introduce a methodological approach that probably has the potential to illuminate category effects, and that has been utilised in report IV of this thesis, namely connectionist modelling. Controlling stimuli for "nuisance" variables The early cases with category specific deficits after brain damage (e.g. Hart, Berndt & Caramazza, 1985; Warrington & Shallice, 1984) quickly caught the attention of a lot of investigators. Not many years after these cases were first described, however, a serious challenge to the very existence of the phenomenon was posed in a study by Fiona Stewart and her colleagues (Stewart, Parkin & Hunkin, 1992). In their report, these authors stressed the importance of controlling for possible differences between category exemplar sets used to demonstrate category specificity. They then went on to show that an apparently clear case of a patient with a category specific naming impairment could perform equally well with animals and inanimate objects when the picture sets were matched simultaneously for three factors that previously have been shown to influence picture naming (namely name frequency, concept familiarity and visual complexity). Funnell and Sheridan (1992) obtained a similar result, by showing that an apparent category specific deficit was explained by variations in Category effects in visual perception 24 object familiarity. Although initially the subject of some debate (see e.g. Parkin & Stewart, 1993; Sartori, Miozzo & Job, 1993), it is now generally accepted that matching stimulus sets on variables that influence picture naming is in most circumstances a necessary precondition for drawing substantial inferences on the basis of category specific deficits. Thus, since the early nineties, all case studies documenting category specific deficits in brain damaged patients have made mostly successful attempts to match stimulus sets across category or domain. Controlling these variables is, of course, equally important when studying category specific effects in 'normal' object recognition. As I mentioned earlier, one possible reason why the 'normal' asymmetry between the recognition of living and nonliving things has been thought to be by some a living advantage and by others its inverse (a living disadvantage), is that the early experiments tended to have relatively poor control over the so-called nuisance variables (Laws & Neve, 1999). In fact, when examining experiments reporting category effects in neurologically intact people, a rather clear picture emerges that seems to support this possible explanation. In Table 1 is a list of experiments, their effects and some potentially confounding variables that were and were not controlled. Despite this apparently revealing list, it was shown in Report I of this thesis that this lack of nuisance variable control may not be the only cause of a living things disadvantage. Still, as numerous studies on the influences on picture naming confirm the effects of variables such as the above (e.g. Barry, Morrison & Ellis, 1997; Bonin, Chalard, Méot & Fayol, 2002; Moore, SmithSpark & Valentine, 2004; Oldfield & Wingfield, 1964; Wingfield, 1968), it seems there is a very strong case in favour of efforts to attain a high degree of control over them. Thus, as far as has been feasible, for the experiments in this thesis we have been employing stimuli that have been well balanced across the domain or category. Category effects in visual perception 25 Table 1. A list of selected studies, their living things effects and control of three potentially confounding variables. Study Living effect Concept familiarity control? Name frequency control? Visual complexity control? Humphreys et al. (1988) Disadvantage Uncertain Yes No Gaffan & Heywood (1993) Disadvantage No No No Lloyd-Jones & Luckhurst (1997) Disadvantage Yes Yes No Laws & Gale (2002) Disadvantage No No No Laws & Neve (1999) Advantage Yes Yes Yes Laws (2000) Advantage Yes Yes Yes Gerlach (2001) Advantage Yes Yes Yes Effects specific to a stimulus set? In much experimental cognitive psychology, where the aim is the testing of specific hypotheses concerning the details of cognitive computations, the socalled "ecological" validity (Neisser, 1976) or generalisability of results is not always the most pressing of concerns; it is often enough to demonstrate a specific effect under very specific circumstances. However, generalisability in one form or another may need to be considered. Especially when one wants to draw conclusions about a type (e.g. objects from a given domain or category), and when the empirical basis for such conclusions can only be had by studying particular instantiations or tokens (e.g. a specific set of objects or object images), one would want to have some sort of assurance that the data obtained Category effects in visual perception 26 from the specific instantiation does not reflect idiosyncrasies exclusive to that instantiation, but rather the nature of the type in general. A majority of experiments and case-studies investigating categoryspecific effects or deficits uses black-and-white line drawings as the stimulus material. Typically, the stimulus drawings are those published in Snodgrass and Vanderwart's (1980) seminal article on picture naming. A very real danger, therefore, is that category-specific deficits or effects observed in case-studies and experiments on neurologically intact participants are specifically related to this particular stimulus set or to line drawings, and that any conclusions drawn about the possible causes of such effects do not generalise to object pictures in general (see also Låg, 2005). Tackling this problem is not as straightforward as it may seem. Although the obvious remedy would be to use stimulus sets from different sources, few such sets are available with a comparable amount and quality of normative data to that of Snodgrass and Vanderwart (1980). However, in order to avoid this problem, a new stimulus set that allowed for control of potentially confounding variables and that consisted of realistically shaded drawings was developed for use in the experiments described in Report II of this thesis. Computational modelling of category-specific effects In the nineteen eighties, computational models built on the principles of artificial neural networks went through something of a renaissance. Although some scientists have worked on artificial neurons and neural networks in earlier decades (McCulloch & Pitts, 1943; Minsky & Papert, 1969; Rosenblatt, 1958), it was not until the seminal work of James McClelland and David Rumelhart and others (see e.g. McClellamd & Rumelhart, 1981, 1986) that neural networks, under the name "connectionism", really came into its own in cognitive psychology. Somewhat later, the relevance of connectionist networks for Category effects in visual perception 27 modelling neuropsychological phenomena also became evident (see e.g. Hinton & Shallice, 1991; Farah & McClelland, 1991; Plaut, 1995). Not surprisingly, then, connectionist approaches to explanations of category-specific deficits in brain-damaged subjects have been implemented in several different models. An early example is Farah and McClelland's (1991) model, in which the ratio of visual to functional (or associative) features of living and nonliving conceptual representations plays a crucial role in the explanation. (This model is thus an implementation of the so-called SensoryFunctional-Theory, first advocated by Elisabeth Warrington's group – see Warrington & Shallice, 1984; Warrington & McCarthy, 1987). Later models have also had their focus on patterns of distribution and association between features. McRae, de Sa and Seidenberg (1997) and Devlin, Gonnerman, Andersen and Seidenberg (1998) both emphasise distinctiveness and intercorrelatedness of features in their models. The work of Moss and Tyler's group is also characterised by this way of thinking (see e.g. Moss, Tyler & Devlin, 2002; Randall, Moss, Rodd, Greer & Tyler, 2004; Tyler, DurrantPeatfield, Levy, Voice & Moss, 1996). A slightly different model by Gale, Done and Frank (2001) is an implementation of the "visual crowding" hypothesis discussed earlier (cf. Gaffan & Heywood, 1993; Humphreys et al., 1988). What is particularly noticeable about all of these different models, is that the sharing and non-sharing of features of concepts (or images in case of Gale et al.'s (2001) model) as implemented in the models' input representations is what drives the category-specific effects the models produce. In other words, the structure of the similarity space generated by the representations is the fulcrum of these networks. This, incidentally, is true for connectionist networks in general. The principles underlying computation in artificial neural networks picks up on the similarity structure of the representations used, and they might, therefore, with a few exceptions, be characterised as "similarity-driven" (see Category effects in visual perception 28 Churchland & Sejnowski, 1992, Chapter 4, for a discussion of the notion of a similarity space and its role in neural network computation). As discussed in the section describing differential interference-effects in the Stroop-task for opponent versus non-opponent colours, there are indications that similarity is a factor that may contribute to this kind of effect (cf. Klopfer, 1996). Connectionist models of Stroop-performance (e.g. Cohen, Dunbar & McClelland, 1990; Kello, Plaut & MacWhinney, 2000; Roelofs, 2003) tend to focus on issues of attention modulation and automaticity, and not at all on issues of differential interference for different kinds of word-colour pairs. However, given the plausibility of a sensory basis of the presumed similarity effects on Stroop interference (see section on 'Category specificity in colour perception'), and given the promise of the "similarity-driven" connectionist networks (also in explaining category specific effects), an attempt to apply the modelling methods of artificial neural networks to differential interference in Stroop might provide some insights. Furthermore, when modelling colour-input, the structuring of colour-representations assumed to have a sensory basis should take the physiological basis of colour perception into account. Of course, an artificial neural network capable of simulating a set of effects does not provide direct evidence that the actual effect is produced in the same manner. However, it does add considerable plausibility to an explanatory scheme if one can show that, when implemented in a concrete computational model, it does behave similarly to research participants executing the same task. The solidity of this support would be increased further if one could show that a removal of crucial assumptions (stipulated by the explanatory scheme) from the implemented model leads to model-behaviour that is not similar to that of research participants. With these considerations in mind, it seemed worthwhile to attempt the modelling methodology of connectionism. Category effects in visual perception 29 GENERAL RESEARCH QUESTIONS An overall concern in the work underlying the present thesis has been to seek clues to the origins of category-specific effects in low-level, visual-sensory factors. This approach, to the extent that it has been successful, does not preclude the efficacy of higher-level factors (e.g. semantic processes) and is not meant to supplant explanations in terms of such processes. Rather, it is meant to broaden our understanding of category specificity, and inform higher-level explanations by illuminating factors that might constitute their sensory origins. The following is a short list of the main issues addressed in the present thesis: i) What is the basis of the of the living-things identification advantage? This question has two sub-questions: a. What role do "uninteresting nuisance variables" play in categoryspecific effects in object identification? b. Can the living-things advantage be attributed to global-shape properties of the 2D representations of objects? ii) Can the use of eye movements research methodology provide clues to the perceptual processes underlying identification of objects and, by extension, to the understanding of category-specific effects in object identification. iii) Can differential (category-specific) interference effects in the Strooptask be interpreted as arising from sensory-level input factors? Category effects in visual perception 30 COMMON METHODS AND DESIGNS The studies and experiments of the present thesis differ considerably in terms of methodological approach. Some commonalities exist, however, and are sketched below. For the particulars of each study, the reader is referred to the relevant sections of the reports. Participants in the experiments were almost exclusively students at the University of Tromsø. Both male and female participants were used, and a reasonable balance between the genders was aimed for. All participants had normal, or corrected to normal vision. Because of the efficiency and sensitivity of within-subjects designs (Keppel, 1991), they were used whenever pragmatic considerations allowed. In Reports I and II, experiments are pure within-subjects designs. In reports III and IV, designs were mixed, but with the theoretically most interesting manipulations as within-subjects factors. In reports I, II and III, object pictures were used as stimuli. The sets used in the three studies do differ somewhat however. For the object stimuli used in the experiments of report I and II, high levels of experimental control over potentially confounding variables were obtained. In report I, the Snodgrass and Vanderwart (1980) set of black-and-white line drawings was used. This set has norms already available for some "nuisance" variables. For the experiments in report II, a new set of pictures was used (more realistic drawings taken from visual dictionaries), for which norms on nuisance variables were collected in a pilot study. For the pictures used in the experiment in report III, practical considerations (the low availability of good 3D models of living things) lead to abandoning strict control of nuisance variables. Analysis of variance (ANOVA) was used to examine the data for effects in the studies of all the reports. This was supplemented by linear regression anlyses in report III and IV. When response times (RTs) were used as dependent Category effects in visual perception 31 variables, extremely deviant responses (> 3 standard deviations above condition mean) were discarded (c.f. e.g. Miller, 1991). SUMMARY OF RESEARCH REPORTS Report I The aims of the experiments in this report were two-fold. First, the role of nuisance variables in category-specific effects in normal participants was examined. Specifically, it was investigated whether or not the living-things disadvantage observed in some early experiments was attributable to poor control of potentially confounding variables. Second, and at the same time, it was aimed at providing some clues to what the factors that underlie such effects might be. Focus was on task demands and stimulus presentation duration. Three experiments were performed, in which participants identified linedrawings of living and nonliving objects. In Experiment 1 and 2, a picture-name verification paradigm was used. Positive trials (name and picture matched) were considered to place low-demands on perceptual differentiation, whereas negative trials (name and picture were not a match) were considered to place higher demands on perceptual differentiation. Presentation times in Experiment 1 were brief (20 ms), whereas in Experiment 2 they were long (1000 ms). In Experiment 3, a naming paradigm was used, and presentation time was manipulated. In all experiments the pictures were balanced across the livingnonliving domains on name frequency, concept familiarity, visual complexity and name length in characters. Results showed that the same set of well-balanced picture stimuli could induce both a living things advantage (Experiments 1 and 2) as well as a living things disadvantage (Experiments 1, 2, and 3). This means that a living things disadvantage is not necessarily a consequence of poor stimulus control. It was also found that in the low-demand conditions of Experiments 1 and 2, there were Category effects in visual perception 32 living things advantages, whereas in the high-demand conditions there were living things disadvantages. This seems to imply that when an identification task requires differentiation between different basic level objects, the task is more difficult for living than for nonliving things, whereas when the task does not require the same level of differentiation, this disadvantage disappears or is reversed. Results in Experiment 1 and 2 were similar, despite very different stimulus presentation times. In Experiment 3, however, the presentation time manipulation caused an effect such that the disadvantage observed for long presentation time was eliminated with short presentation time. It seems, then, that presentation time may influence category-specific effects, but only when, as was presumed to be the case in Experiment 3, other task demands do not impose too strong constraints.